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ABSTRACT 



The aim of this project was to develop effective techniques for identifying educational 
needs and formulating them into well defined problems. The critical incident technique 
was used to identify need symptoms perceived by educators for a representative sample 
of schools throughout the region served by the Far West Laboratory . A hierarchy of 
categories was developed to describe these need data, which were then classified 
and tabulated by category. The problem formulation techniques explored informally 
included interview, observation, and small group discussion. We decided that 
knowledge of problem defining behavior is too primitive to justify step-by-step 
systemization of the process of formulation. The techniques we developed rather 
relied heavily on intuitive processes, and systematic control was aimed toward 
insuring that the product of the formulation contained the following elements: outcomes 
desired, values underlying outcomes, kinds of evidence for outcomes, present 
conditions, solution possibilities and immediate action alternatives. We experimentally 
compared three group problem defining techniques which differed on two dimensions: 

(a) whether or not the formulation was structured to obtain the elements listed above, 
and (b) whether or not the inquiry was directed by an experienced outside formulator. 
Eighteen three-person groups of educators were used in all . The problem definition 
produced by each group was evaluated independently by two other educators from 
the same district. Results favored the directed groups; among the undirected groups 
completely unstructured groups rated as high or higher than groups producing a 
structured definition. Implications for training and promising directions for further 
work on problem formulation systems are discussed. 
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INTRODUCTION 






A teacher and his principal have different views on discipline procedures for 
controlling tardiness and truancy. The teacher feels that he spends all his 
time running a prison and doesn't get much teaching done. The principal 
feels that the teacher wouldn't even have any students if strict control were 
not maintained, and recalls that a more open policy tried in the past resulted 
in massive truancy, tardiness, noise and vandalism in the school halls as well 
as on the city streets, which produced complaints by citizens, etc. The 
superintendent feels obliged to respond in some way. 

Situations, similar to this, occur in many school districts. However, what 
happens next depends a great deal on how the superintendent defines the 
problem. For example: 



Superintendent A sees a teacher making unrealistic demands to suit 
his own convenience and tries to select more cooperative teachers in 
the future. 



Superintendent B sees an over-militant principal and looks for ways to 
create a more campus-like atmosphere at the school. 

Superintendent C sees the problem as one of conflicting personalities 
and looks toward transferring one to a different school. 

Superintendent D sees a communication problem and tries to set up 
regular channels of communication between the principal and his 
teachers. 




Superintendent E sees that the principal is rationally pursuing one 
objective while the teacher is just as rationally pursuing a different 
objective which conflicts with the first. 

Each superintendent will be exploring a different set of solutions because of 
the particular way he defined the problem. The man who picks a problem 
definition well suited to the situation may find several effective solutions 
to choose from. The man who defines the problem less aptly may find ibat 
the best alternative action available is inadequate. 

Solving problems and making decisions are widely recognized as basic and 
important processes to be studied scientifically and improved, in education 
as in other areas. A good deal of theory and research has focused on how 
to choose best among the available alternatives once the problem is clearly 
defined. Cost effectiveness strategies, computer simulation of problem solving, 
and theories of games, decisions and utility inevitably start with clearly de- 
fined goals and alternatives (Luce & Raiffa, 1957; Newell, Shaw & Simon, 
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1958; Feather, 1959; Me Kenney, 1967). Experimental studies of problem 
solving have, to our knowledge, all started with well-defined goals, whether 
or not the alternative actions were well defined, (e.g., Rimoldi, 1960; 
Hemphill & McConville, 1962; Kleinmuntz, 1966). Yet very little is known 
about how a problem is defined in the first place, even though this initial 
definition probably has more effect on the eventual success or failure of the 
decision than anything else. This project represents a start in the direction 
of trying to improve the indentification and clear formulation of educational 
problems. 

Project Objectives 

The main objective was to develop one or more ways to identify educational 
needs and articulate these into well defined problems which can be attacked 
directly by research, development or social action. A second objective was 
to describe the educational needs and problems currently being encountered 
by elementary and secondary school districts in the Far West Region. 

Overview of Procedure 



1. During the first six weeks of the project, we outlined a plan of 
action for the remaining eight months and suggested a number of 
possible approaches to identifying educational needs and formulating 
them into problems in an interim planning report (Campbell, et.al., 
1967). 

2. We collected a broad sample of need symptoms recalled by educa- 
tors throughout the Far West Region, and classified these into need 
categories. 

3. We reviewed the literature on problem definition and spelled out 
specific criteria that could be used to judge the adequacy of the 
formulation of a problem. 

4. We explored particular problem defining actions, largely through 
individual interviews. From this we developed a set of key prob- 
lem elements which served as a guiding framework for subsequent 
problem defining techniques. 

5. We compared three different small group approaches to problem 
definition in four school districts in the region. The approaches 
differed in the extent to which discussion was structured and 
channeled toward the a priori set of key problem elements. In 
two of these districts, "additional interviews and observations were 
conducted to check the strengths and deficiencies of the small 
group approach. Evaluations of the products of the problem 
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defining group effort were obtained from most of the participants 
and two independent evaluators in each district. 

NEED IDENTIFICATION 

Identifying a need and defining if clearly are closely interlocked processes. 
However, for purposes of analysis the two processes can be divided con- 
veniently as follows: The initial act of calling attention to some area of 

concern will be called need identification. After a need has been initially 
identified any attempts to clarify the nature of the need or to spell out 
alternative courses of action will be called problem formulation or problem 
definition. 

There are many reasonable procedures for identifying educational needs of a 
school district. Educators within the district may do so on the basis of their 
recent experience. Outside experts, or educators within the district, may 
collect new data as a basis for identifying needs. The data may involve 
observation, questionnaires, checklists, interviews, and inspection of docu- 
ments among other things. In order to achieve our second objective, i.e., 
to collect a coherent body of data on actual current needs in the region, 
our effort at need identification focused on a single promising technique 
which had not been previously tried in this context, the critical incident 
technique. 

The Critical Incident Approach to Need Identification 

The process by which needs are identified has usually been intuitive, even 
in recent years. Most commonly, a leader in education, science, or public 
life calls attention to some deficiency in education after an unknown process 
of observation and thought. Occasionally the assertion is based on explicit 
research findings. 

There has been little scientific research on the identification of educational 
needs. Most systematic efforts at need assessment have been generated by 
recent governmental programs of research and development in education which 
require that need assessment be part of initial planning. These studies have 
usually involved asking people with various slants on education to make a 
direct judgment of the degree or presence of a need. These are useful data 
since efforts to meet needs depend upon their being perceived by persons 
who have power or information relevant to the needs. An interesting matter, 
however, is to what extent such judged needs are based on hearsay and the 
general hue and cry, rather than upon experienced failures in the school's 
operation. 

We chose the critical incident approach to identifying needs in order to 
bring to light the specific, concrete experiences of educators on which, we 
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assume, judgments of felt need are based. The critical incident technique 
(Flanagan, 1954) was developed mainly to identify good and poor performance 
on the job by focusing on the most memorably effective and ineffective be- 
haviors of a particular person in that position. The present application of 
the technique identified m^ny kinds of events within a system, without re- 
stricting the data to behaviors of a single person, although many of the 
events were in fact individual behaviors. Educators were asked to recall a 
specific event or condition observed recently which made them feel that 
something about their educational system needed improving. 

The Sample . In order to obtain a representative sample of need symptoms, 
as we will refer to these recalled events, we first spelled out the variables 
differentiating schools in the Far West Region that might be associated with 
the greatest differences in the specific experiences and felt needs of educa- 
tors. The following variables were considered in sampling: geographical 

area, population density, size and type (elementary, secondary, unified) of 
school district, socio-economic level, and racial composition. The twenty- 
three school districts and county offices which provided critical incident 
data were handpicked to be representative of the region as a whole on the 
above variables, but without any prior knowledge of the particular educa- 
tional needs which might be identified. 

The number of educators requested to participate in each district was roughly 
proportional to the size of the district. The administrator within ths district 
who arranged for the data collection was asked to choose a variety of teachers, 
administrators, counselors, coordinators, consultants aiid persons in other 
positions so the sample would be approximately representative of the dis- 
tribution of educational positions in the district. 

Participants convened within a school or district to complete the critical in- 
cident forms which are shown in Appendix A. The form requested four in- 
stances of symptoms of an educational need, and one instance of an event 
which indicated a need well met. The following instructions were given to 
the participants: 

1 . The purpose of the project and potential value to schools 
were outlined. 

2. Confidentiality and anonymity were assured. 

3. The need for specificity of incidents was emphasized. 

4. Incidents were invited from any aspect of education, 
including, but not limited to, student performance, 
administration, guidance, instruction, learning, behavior, 
curriculum, human relations, or communication. No 
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examples of incidents were given because of the known 
channeling effect of providing examples. 

The Nature of the Reported Incidents. The critical incident form requests the 
respondent to try to remember a specific recent event. However/ many of 
the reported problems are more general in nature than a specific incident/ 
summarizing either a set of events or a generalized feeling. Interviews with 
respondents suggested that they felt that the data collection effort could not 
profit as much from the trivial details of everyday existence as it could from 
responses which summarized these details. Some respondents viewed with 
disbelief our request for simple narrative descriptions, thinking we might be 
disguising a different research interest. Thus there is a large amount of 
variability in the type of report received. Some reports are, indeed, 
critical incidents, such as the arrest of a student for selling drugs. Others 
are far more general and are not properly called "incidents", clthough that 
term will be used here for convenience. Among the wide range of responses 
are system critiques, worries about societal trends, and questions about the 
proper function of the school system. Data of these types do not lend 
themselves to simple classification in such terms as principal agent, location, 
effect, etc., as might a set of incidents which were all at a simple 
operational level . 

Basic Classification. The negative incidents, or need symptoms, have been 
classified in terms of the major stated or implied locus of the difficulty, 
as reported by the respondent. Although project staff were tempted to 
reinterpret some of the reports ("This one really seems to be a problem with 
x, not y"), every attempt was made to avoid reinterpretation, in order that 
the classification would represent educators' perceptions of problem areas, and 
not those of the staff. A second intent of this restriction to what the 
educator explicitly said was to minimize the amount of inference needed in 
classifying the incidents. 

The complex nature of the problems often made unitary rather than multiple 
classification somewhat arbitrary. For example, a criticism of allocation of 
funds would almost always include an explicit or implicit regret about the 
insufficiency of funds in general. If general insufficiency was not mentioned, 
it would be classified under allocation. If both were mentioned, it would be 
classified under the more strongly emphasized one. 

There is a similar nesting of categories in the case of problem children and 
their parents. Some reports clearly stated causal hypotheses about deviant 
or inadequate parental behavior being responsible for deviant child behavior. 
The stated locus of the problem is clearly the parent in these cases, and the 
incidents were so classified, even though student behavior was cited also. 

In other cases, however, no parental reference was included to explain deviant 
child behavior. In the absence of such reference, the child himself was 
classed as the stated locus of the problem, even though the inferential leap 
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to the parents' behavior would be very short in many cases. 

Systematic reliability checking of the categorization was not done, but the 
development of the categories was done in several parallel independent 
stages, with rater disagreement used to modify categories. The categories 
were developed inductively from the data, as described in a later section. 
Reliability is affected adversely both by the multiple nature of the problems 
and by the large number of categories. But a large number of categories 
was retained in order to describe the data more fully. 

Development of the Categories. An initial reading of several hundred 
incidents revealed that the majority of the negative incidents fell into two 
basic categories: those in which the focus was on the behavior of specific 
individuals, and those in which the focus was on the functioning of some 
aspect of the overall system. These two broad categories can generally be 
distinguished in terms of potential corrective action. The incidents in the 
individual (I) deviation category suggest changing the behavior of the 
deviant individual or perhaps replacing him. Incidents in the system (S) 
category suggest a change in policy, procedure, equipment or some feature 
of the system other than the particular people in it. Some incidents imply 
a change in the relation of the school system to o^her systems with which 
it interfaces, such as the community, publishers, or the state. Of course,, 
many system changes would be easier to implement with a corresponding 
person change, and some person changes would result in system changes, 
but the categorization scheme does not explicitly deal with this kind of 
inference. Examples of incidents in each category are given in Appendix B. 

Individual Behavior . The broad " I " category was first subcategorized logically 
in terms of the roie of the person or persons being critically evaluated. These 
are shown as major outline headings in the listing of 1 categories (Table 1) 
and are II, Student; 12, Teacher; 13, Specialized School Personnel; 14, 
Administrator; 15, Parent. Each of these categories was then subdivided 
inductively according to the kind of criticism being leveled at the 
individual (s). The rationale for the subcategories varies from main category 
to main category, according to how the incidents were found to cluster. 

Thus the subcategories for students ( II .1, II .2, etc.) vary according to 
the severity and effect of the deviant behavior, while the subcategories for 
teachers ( 12.1, 12.2, etc.) and administrators ( 14.1, 14.2, etc.) vary by 
task function or area of competence. 

System Function. An initial attempt was made to achieve symmetry between 
the system (S) and individual (I) categories by making the subcategories 
parallel. This was successful for a subset of the incidents, but the scheme 
awkwardly crosscut a number of categories which were developed inductively, 
directly from the set of "S" incidents. Attempts to keep the two breakdowns 
parallel were eventually dropped and the current scheme evolved. Overlaps 
in the categories and complexity and generality of many of the incidents 
pose more of a reliability problem in the "S" category than in the "I" category. 
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SI through S5, i.e.. Curriculum, Instructional Methods, Guidance and 
Placement, System Expansion and Teacher Competence^, all relate basically 
to instruction-related classroom activities. The separation between these 
categories is clear in most cases. However, a problem which involves 
improving teacher competence in order to use new methods to introduce a 
modified curriculum in an expanded system which uses more sophisticated 
placement could fit in any or all of these five categories. Fortunately for 
classification purposes most respondents did not enumerate procedural and 
causal chains in this detail, so classification on a non -inferential level could 
be carried out reasonably easily. This form of classification identifies an 
issue within a broad category (e.g. S1-S5, inclusive) then pinpoints saliency 
for the specific respondent. 

The interpretation of S6-S12 is less direct. Each of these appears to be a 
discrete issue, but because communication is on a different level of analysis 
than funding, for example, it is difficult to relate the two. Communication 
difficulties could involve any of the other categories in this analysis. As 
with the other categorization problems, specific attributions of the locus of 
difficulty were used as much as possible to derive these categories. Sub- 
categories were derived similarly. 

Positive Incidents. There are clear parallels between the positive and negative 
incidents, although the smaller number of positive incidents led to the 
development of fewer subcategories. It should be noted that the smaller number 
of positive incidents was a function of the data collection procedures (four 
negative forms, one positive form per respondent) and not of respondent bias 
or preference. Occasional respondents failed to answer the positive page 
(the second) and a few mistakenly provided a negative incident on that page, 
but the bulk of omissions were of the third or fourth negative incident — 
usually a function of time pressure. Because the positive reports tended more 
often to be actual incidents, less difficulty was encountered in deriving the 
categories and coding. The same inductive procedures were used. 

Frequency Tabulations. The incidents have been tallied in Table 1 by 
incident categories of all levels. They have been cross tabulated according 
to the job classification of the respondent. One can see at the top of 
Table 1, for example, that a total of 1042 incidents were collected. Of 
these, 809 were negative. Looking down at the 12 category, one can see 
that of the 86 criticisms of individual teachers, 35 were made by teachers 
and 34 by administrators. Comparing these in ratio to the total number of 
“I" incidents contributed by teachers and administrators, it can be seen that 
administrators contributed approximately twice as many such criticisms in 
proportion to the marginal totals (3^100 vs. 35/217). The raw frequency 
data in the 14 category suggest the expected reversal of this distribution, 
with 75 negative incidents about administrators provided by teachers and 28 
provided by administrators. The ratios do not bear this out as strongly as the 
raw frequencies suggest, however — 75/217 vs. 28/100 or .35 vs. .28. 
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Comparisons between respondent groups must take these ratios into account. 
Within or across respondent groups, the raw frequencies can be used directly. 

A Comparison with PACE Center Need Assessment Data 



It is difficult to make a direct comparison of the tabulation of incident data 
with the needs identified by the Title III Supplementary Education Centers for 
several reasons. The needs reported by the centers are summaries of data 
which have been gathered in a wide variety of ways, so a single methodological 
contrast cannot be made. Some PACE center efforts involved questionnaires 
which were far more structured than our incident forms, while others used 
much more freewheeling conference/consensus techniques. 

A fundamental difference between the PACE reports and this summary of data 
is the difference in level of generality. The PACE needs are generalizations 
which have been derived from data. The incidents and needs reported here 
have not been summarized in this way, because the intent has-been to 
identify symptoms of needs rather than needs themselves. Also, some of the 
PACE data have been deliberately given priority ranking, while only a tacit 
set of priorities can be derived from the frequencies in the data in this 
report. 

There are, of course, parallels between PACE needs and the incident categories, 
just as there are parallels from PACE center to PACE center. Curriculum, 
guidance, and social problems all receive strong emphasis in both. But there 
is a clearcut difference in general content between the PACE summaries and 
the incident data. The incident data reveal a much stronger focus on 
internal system operation, management, and personnel practices. Of \46 
reported needs from 18 county PACE centers (with many overlaps of needs) 
only one reported need involved such internal system issues. Clearly the 
focus of the PACE effort is on external system goals, as the majority of the 
needs are stated in terms of broad effects of the school system on the students. 

Potential Uses of the Incident Data 



One of the initial reasons for collecting the incident data was to identify the 
potential range of problems which a problem formulation system might encounter. 
A second was to get examples of the form which initial inputs to the system 
would take if educators were asked to provide observational data. One type 
of potential use of incident data is to classify and store it at a regional, 
state or national information center to be used as a resource by problem 
formulators or for other research purposes. This is discussed further in the 
final section of the report. 

Another more direct use of the technique is to identify needs in a given 
school system as an input to problem formulation. Judging by the wide 
variation in the incident data we obtained it is clear that an important 
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initial task of a problem formulation system is to ascertain whether an incident 
reported identifies one need, none, or many. Although a few respondents 
reported an incident with no apparent implication beyond, "That's the way 
life is," most incidents pointed clearly to at least one need. Where many 
needs are identified the problem formulation system may need a criterion 
for selecting which most deserve defining clearly, or else a classification 
system which indicates how different types should be handled. One might 
imagine an initial screening routine which would route different perceived 
needs to different formulation subsystems. Decisions about what needs to 
exclude, and what the division of subsystems might be should be made with 
knowledge of the general economic structure of the system. The sources of 
funding, questions of who initiates the formulation interaction and estimates 
of potential payoffs are all relevant to such decisions. 



PROBLEM FORMULATION 



A survey of the educational literature on techniques for defining problems, 
as distinct from identifying needs, reveals very few studies, and these tend 
to treat problem definition as a single step in problem-solving (e.g., 

Schmuck, et al., 1966). Literature on defining problems comes mostly from 
other disciplines. In clinical and industrial psychology attention has been 
given to the interview as a technique for defining personal or interpersonal 
problems (Sullivan, 1954; Maier, 1958; Kahn & Cannell, 1958). The study 
of that aspect of industry which has to do with maintaining complex machine 
or man-machine systems has evolved a problem formulating technology called 
troubleshooting. However in troubleshooting, as in medical diagnosis (e.g., 
Rimoldi, 1960), the problem is already partially defined in that the goals are 
clear. The fields of business management and public administration, too, 
have apparently been struck by the frustration of trying to solve problems 
before they are well defined (Barnard, 1938; Simon, 1950; Griffiths, 1959; 
Markle, 1967; Odiorne, 1965; Kepner, 1965; Timms, 1967). 

As noted earlier most studies of behavior "about problems" concentrate mainly 
on solving problems rather than on defining them. However, some of this 
research does delve into the prior process of problem definition, especially 
those studies which involve "intensive tinkering" with the creative process. 
Mednick (1962) inferred three important heuristics for having good ideas 
about problems: serendipity, similarity, and mediation. He hypothesized 

that massed practice on a problem would work better because the intensive 
thought over a considerable length of time would enable more remote associa- 
tions to yield new ideas. The "Synectics" group at Harvard (Gordon, 1961) 
found it fruitful to make familiar things seem strange by developing analogies 
regarding them and thinking about them metaphorically. Maier and Solem 
(1962) found that "problem mindedness" was more effective than "solution 
mindedness" in generating better solutions to problems. They inferred that 
problem orientation increased the number of alternatives considered, which 
increased the quality of problem solutions. 

Many studies (e.g., Duncker, 1945; Scheerer, 1963) have found that func- 
tional fixedness or stereotyped thinking regarding solutions interferes with 
effective problem solving. However previous experience may also have a 
positive transfer effect or no effect at all, depending on its relation to the 
problem to be solved (Jensen, 1960; Gibbons, 1965; DiVesta and Walls, 

1967). It is reasonable to expect that transfer in defining problems should 
likewise vary as a function of the similarity in the structure of the problems. 

An empirical approach to defining educational problems might be to observe 
how it is now being done, either by typical educators or by those reputed to 
be especially good or poor at defining problems. Then one could combine the 
more promising practices observed into some overall procedure. Another approach. 
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often identified with systems analysis, is to begin by spelling out the 
objectives of the educator or educational unit which feels a need. From 
these objectives the problem formulator may then deduce problems and 
needs by comparing the logical implications of these objectives with 
characteristics of the existing system. In this way he may well wind up 
defining problems or needs that had not yet come to the attention of the 
educator within that system at all. A program is currently underway in 
California to develop use of the system analytic approach in educational 
planning by local and state educators (Miller, 1967; Johnson, 1966). The 
empirical and logical approaches may be combined in many ways, of course, 
and our own effort to develop problem formulation techniques represents one 
of these ways. 

Criteria for the Adequacy of Problem Definitions 

Although things got worse later, we started off in trouble because defining 
a problem is not a well defined task. One of our primary aims therefore 
was to spell out reasonable criteria for judging when a problem is well 
formulated. Criteria will be listed and discussed under three headings: 

(a) the important parts or kinds of content which a complete problem 
definition should logically include; (b) characteristics of all parts of the 
content which are expected to make the formulation more effective; (c) 
practical consequences of having a problem well or poorly defined. 

A. Rational Elements of a Defined Problem. No matter who translates an 
identified need into a well defined problem, nor by what process it is done, 
we assume the results must be a recordable communication to those who are 
concerned with acting on the problem. In elementary and secondary education, 
virtually every need requires a joint decision and/or a joint action by two or 
more persons. We therefore assume that in practice a good problem definition 
would be recorded in some way so that it could be referred to, re-examined, 
and perhaps revised during subsequent processes of action on the problem. 

This assumption enables us to apply our criteria for problem definition to a 
tangible product in every case rather than relying on evaluation of a 
process. By contrast, defining and solving a private personal problem might 
take place effectively without any observable, recordable product. 

The question arises whether all well defined problems encountered by 
elementary and secondary educational systems have certain essential elements 
in common. We have encountered no experimental evidence indicating 
whether the inclusion or exclusion of certain elements in a problem 
definition matters to its subsequent effectiveness. The discussion of each 
element in the list below is thus confined to a rational analysis of its 
importance . 

1 . Outcomes Desired . If a need exists, then something is unsatis- 

factory and needs improvement. Knowing what is wrong about the 
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status quo does not necessarily tell what state of affairs would be 
considered right. The desired results need to be spelled out explicitly. 

If more than one outcome is sought, priorities or relative importance of 
the outcomes need to be specified as a basis for later choices between 
alternative actions which serve different outcomes. 

2. Reasons for Valuing Outcomes. One common type of regret or 
failure which can sometimes be attributed to inadequate problem 
definition is that in which achieving the outcomes directly sought 
results in the side effect of losing some other more important goal . 

For example, a series of disciplinary policies and actions designed to 
reduce the amount of noise and distraction in classrooms may effectively 
do this and at the same time have the overriding bad effect that 
students fee! so oppressed that they no longer inquire or seek to. learn 
on their own. Spelling out the values affected adversly by achieving 
the specific outcome sought, as well as those values served positively, 
may well change the educator's priorities among outcomes. 

What about personal ulterior motives for seeking particular educational 
outcomes? These may have important effects on educational decisions 
made by individuals and must be taken into account if we are seeking 
ways to define and solve the problems of individuals. However, since 
the concern here is with the problems of educational bodies which 
have a public responsibility, from both an ethical and a practical 
standpoint we can safely omit reasons and values which are not 
publicly justifiable. It is appropriate to examine such personal motives 
as aids or hindrances to achieving the desired outcomes, but not as 
ends in themselves. 

3. Evidence of Outcome Achievement . It often occurs that after an 
innovation has been tried by a school system for a year or two there 
is a prolonged debate over whether it is working or not . This could 
be avoided by agreement in advance on what will be accepted as 
evidence that the desired outcome is achieved. Requiring that 
evidence of outcome achievement be spelled out has an important 
consequence in addition to knowing whether you're getting there or 
not. Outcomes or objectives are often stated in general, vague 
terms. I Agreeing on evidence often forces the problem formulators to 
be moA? specific about the outcomes sought and thereby to communicate 
more cjlearly. The benefit works in the other direction too. 

Specifying outcomes separately from their evidence helps to avoid 
"teaching the test". If the objectives are lost sight of and the 
evidence becomes an end in itself the result may be sham achievements, 
such as students who know the big words but can 't apply them 
usefully, or puppet student governments. 
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4. The Present Condition. Sometime before the stage of proposing 
action one must assess the extent to which the outcomes desired are 
already being achieved. A good deal of what might be brought up 
in describing the present condition and its context might in retrospect 
be considered irrelevant distractions from getting at the heart of the 
problem. Yet it is difficult to know in advance which conditions will 
turn out to be relevant and which irrelevant. 

One class of conditions which is of uncertain value as an input to 
the problem definition process is that of "causes" of the problem. 

Good definitions and solutions to problems are indeed often obtained 
by identifying causal factors and proposing ways to remove them. 

On the other hand, problem formulation may result in restructuring the 
whole problem so that outcomes are sought by new approaches which 
totally disregard the causes of the problem as it was initially perceived. 
For example, a need initially identified as getting students to turn 
their homework in on time might be attacked by examining causes 
of their laxity and trying to remove them. On the other hand, 
spelling out objectives (e.g. "that students learn all they are capable 
of in a subject matter") might lead to dropping homework deadlines 
from the list of outcomes desired • 

Some aspects of present conditions are essential as background for 
evaluating proposed actions. One such kind of information consists 
of limiting factors such as budget, personalities, laws, policies, etc., 
which will have to be accounted for regardless of what action is 
chosen. One of the advantages of spelling out these limiting factors 
is that they may be challenged if they are explicit but not if they 
remain as unspoken premises. When stated in words a way may be 
found to overcome such obstacles by assuming different premises. 

Another valuable kind of background information is the history of 
previous actions taken to meet this need and the results of such 
actions. 

5. Possible Courses of Action. Problem formulation may be defined 
to include spelling out alternative possible actions or to exclude it 

on the grounds that it is part of solving the problem rather than defining 
it. Our reason for including it is that people are strongly inclined 
to discuss solution ideas, whatever phase of problem definition they 
may nominally be involved in. As will be discussed later, this may or 
may not contribute to effective problem definition and solution, but the 
fact is that it is hard to avoid entanglement of these aspects of the 
total analysis of the problem when you are using real human problem 
solvers without suppressing their activity altogether. We also include 
evaluation of the pros and cons of any suggested action as part of the 
process of problem formulation, though we stop short of including any 
actual decision or choice among alternatives. The phrase "courses of 



action” is meant to include taking no action, getting information 
through search or research, immediate steps, contingent sequences of 
action, and long-run solutions or programs. 

B . Effective Characteristics of the Content of a Problem Definition. The 
following criteria are applicable to the content of any problem definition, 
whether its parts are the kinds described above or not. These criteria are 
internal like those above in that they do not refer to consequences of 
problem definition but to the definition itself. The importance of these 
internal criteria is that they may be used immediately on completion of the 
problem definition, as was necessary in the present research. 

1 . Number of Relevant Ideas and Facts. The problem definition may 
be considered to be a resource for the decision maker. If it is a 
resource and not a constraint, then the decision maker is free to reject 
as much of it as he pleases. From this standpoint the average quality 
of the ideas in the document may not be as important as the total v, 
number of good ideas and relevant facts. Since it is usually hard 

to know whether an idea is good or not until it has been tested, the 
total number of relevant ideas is a practical criterion which does not 
require dealing with the slippery criterion of quality. 

2. Quality of Ideas. As difficult as it may be to measure quality 
in a satisfactory way, nearly everyone seems to think that quality is 
close to the heart of the matter and cannot be ignored. It is the 

. characteristic which comes closest to being a prediction of the potential 
effectiveness of the formulation in later decision-making. Quality 
measures can of course be applied to the individual ideas in a 
statement or to the total problem definition. An interesting approach 
to measuring the quality of the total definition is to ask someone 
familiar with the problem or similar problems to list the critical 
questions about the problem which are yet to be asked or answered 
by the problem definition. Of course the number and kind of such 
questions can be expected to vary with the type of problem and the 
repertoire of the evaluator. These sources of variation might be 
controlled to some extent by categorizing the most critical questions 
which are typically asked and providing the evaluator with the categories 
or even the questions as a check list. 

3* Relatedness of Parts. The worth of relating outcomes to values and 
to evidence of these outcomes was discussed above. It is equally 
necessary that the other parts or elements of a problem formulation be 
closely related in order to keep the formulation on track and also to . 
stimulate new ideas through the conjunction of parts. As an example 
of the latter, identifying symptoms of the present condition may suggest 
new kinds of evidence to be used in measuring achievement of the 
desired outcomes, and vice versa. Keeping on track is a critical 
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problem in view of the very short attention ond memory span of human 
beings. When the parts of a problem definition are developed separately 
it seems to be quite easy for them to become unrelated or even 
inconsistent. For example, solutions may be proposed which seem to be 
promising for some purpose but do not directly serve the top priority 
objectives. Turner and Fattu (1960) concluded that consistency was a 
more useful criterion of problem solving proficiency in elementary 
school teachers than were consensual or ultimate criteria. 

4. Clarity and Precision determine the communicative value of the 
problem definition. In addition, clear, precise statements of the 
problem are more likely to facilitate decision-making than are 
ambiguous statements. 

5. Feasibility. Are the goals attainable? Do proposed actions lie 
within the range of capability of the educational system? Are costs 
related to and bounded by importance of outcomes? 

The question arises as to who should apply these largely judgmental criteria • 
Educators within the system experiencing the need are more likely to be 
familiar with all aspects of the problem and to be concerned about the 
effectiveness of the definition of the problem . Educators from other systems 
are not as likely to be biased from their previous involvement in the 
problem, and may have had experience with similar problems. Experts who 
specialize in j^roblem formulation and evaluation might serve best, but it is 
not now clear whether such persons exist. 

C. Practical Consequences. Since problem formulation is a means, not an 
end in itself, ultimately the most important criteria for success are its 
practical consequences for subsequent planning, decision-making and action 
in the educational system. Unfortunately these criterion measures are also 
among the most inconvenient to obtain. There is often a considerable delay 
before a formulated problem is acted upon and a further delay before the 
consequences of such action can be evaluated. Another difficulty is that 
the quality of the problem formulation is only one of the factors in the 
complex set of determinants of later actions. 

1. Communicating the Problem. One of the most immediate practical 
consequences is communicating about the problem. If those who define 
the problem cannot make themselves understood to others concerned, it 
seems unlikely that any other good will result from the definition. 

2. Consensus. If planning and action are to fxoceed effectively, 
there must be substantial agreement on what the problem is among 
those persons involved in the planning. If the problem is well defined 
and the statement communicates well, it should be easy to locate and 
isolate the specific points on which there is disagreement about the 
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nature of the problem. These specifically defined differences of 
opinion may then reasonably lead the group to plan research as o 
preliminary step rather than to immediately implement a program for 
solution, if the outcomes are important enough to warrant the cost of 
seeking more information. 

3 . Effects of the Formulation on the Following Decision Process . The 
degree of consensus on a decision or plan of action should reflect in 
part the adequacy of the preparatory problem formulation. However, it 
will also reflect the extent to which one plan clearly has more merit 
than other alternatives. If two or more alternatives have about equal 
merit for achieving the desired outcomes, consensus will be less likely 
regardless of how well the problem is formulated. 

Two other process criteria which need to be considered in conjunction 
are: (a) time required to reach a decision, and (b) the number of 

alternative actions seriously considered. In general we would expect 
that the more solutions that are considered and the less time it takes 
to consider them, the better the problem formulation. Combining 
these two criteria into a ratio (e.g. number of alternatives considered 
per hour) helps avoid the extraneous effects of variation in overall 
time and effort given to a problem as a function of its importance. 

Like consensus on a plan, these criteria are also affected by the 
equivalence of alternative plans, in ways that are hard to predict. 

4. Success of Action. This type of criterion comes closest to being 
the final payoff of good problem formulation. The criterion can be 
an absolute measure of the extent to which specific objectives are 
achieved, or it can be a relative measure indicating how nearly the 
best of all possible alternative actions was chosen. A practical, 
relative measure might be the number of regrets expressed by educators 
in the system that some other foreseeable alternative was not chosen. 
Absolute measures are best applied to specific, quantifiable objectives 
such as a one-year increase in average reading grade placement level. 
Less direct measures of success are increase or reduction in rate of 
occurrence of problems similar to the one formulated, and the 
satisfaction that the need has been met expressed by parents, citizens, 
board members, educators, students or other concerned groups. If 

a problem formulating system were applied to most or all of the 
needs encountered by an educational system during a given period of 
time, it would be reasonable to assess the effectiveness of the problem 
formulation system by some estimate of the total cost effectiveness of 
the school system in meeting its objectives. In most school systems 
this would require as a first step spelling out the educational objectives 
of the system. 
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Exploratory Problem Formulation Interviews 



Our strategy was to explore through individual interviews what functions should 
constitute problem formulation before trying to maximize the social arrange- 
ments by which these functions are achieved. Early in our exploration we 
found that problem formulation is a many-splendored thing • So much so 
in fact that we were overwhelmed by the number of alternative directions 
the process of definition can take. The most invariant features of the 
entire set of possible processes seemed to be the component parts of the 
product which might result from the process. We therefore chose to 
structure our development of problem formulation techniques around obtaining 
key elements or parts in the product itself. More specifically, we sought 
to confirm and refine the rational elements of a problem as discussed above 
and to decide how the process of obtaining these elements should be guided. 

Do all well defined problems share the same basic elements? We approached 
this question first by role playing interviews in which one staff member 
played the problem formulator and another staff member played the role of 
an educator experiencing a particular need as described in one of the 1,000 
or so critical incidents obtained earlier. The four staff members involved 
switched roles occasionally and tried this procedure with 20 or 30 different 
critical incident cards as source data. We then individually interviewed 
ten teachers and administrators from nearby school systems. From these 
explorations we concluded that there are about five basic elements (those 
described above) useful to the definition of any problem arising in an 
elementary or secondary system. The most undependable of these five 
elements was "the present condition”. The amount of relevant information 
concerning causes, history of previous actions, limiting factors^ etc., varied 
considerably in kind and quantity. 

We wondered whether it mattered in what order the basic elements were 
obtained, but for the most part it did not appear to matter much. We 
had supposed that too early a consideration of solutions might cause closure 
of thought too soon. Maier (1967) found that in problem solving groups, 
closure, commitment or decision tends to be remarkably stable and 
predictable from the number of valuative statements made about solution 
alternatives. That is, a decision tends to be reached about 85% of the 
time when the number of favorable statements exceeds the number of 
unfavorable ones by 15, regardless of the group size or problem. However, 
the problem formulation situation is somewhat different. It is easy for 
the formulator to redirect attention from a solution to objectives by 
merely asking why a proposed solution is wanted. If the goal is limited 
to defining the problem, no one expects a decision to be made, so there 
is little pressure against returning anew to a discussion of goals. For 
this reason, having problem formulation occur at a separate earlier time 
than decision on action may be a good technique for preventing 
premature closure on a particular solution. 
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In problem formulation early discussion of solutions may thus waste some time 
if the solutions are not germane to the outcomes finally selected as 
important* This does not seem like a serious drawback, and since people 
are so inclined to think about means and actions and may even be thus 
stimulated to think better about other aspects of the problem, curbing 
their natural tendencies to do this hardly seems justified. 

During our early interviews, we started to list all reasonable steps a 
problem formulator might take in order to obtain the basic elements, and the 
cues which would indicate when to take these steps. A partial list appears 
in Figure 1 . The intent was to develop such a list into a check list that 
could be used during problem formulation to systematically guide what has 
heretofore been a completely intuitive process. However, it soon became 
apparent that a reasonably complete listing of feasible steps and cues would 
be too complex to guide an ongoing discussion. If a computer were to 
guide the formulation process the logistics of this approach would be more 
feasible. However the fundamental weakness which this would not solve 
is that too little is known about what actions are really most appropriate 
and what their cues should be in order to obtain the key information 
needed . 

This brings us to the most critical issue of all in developing a system for 
problem formulation. Unfettered intuition is still probably the most 
effective way to define a problem. A system which spelled out in detail 
the steps to be rigidly followed by a formulator would undoubtedly interfere with 
this intuitive process. Until that remote day when the appropriate problem 
formulating steps can be spelled out exhaustively for all types of educational 
problems, a practical problem formulation system should make the most of 
intuition, rather than try to suppress it. Probably the main failing of 
intuition as a guide to problem formulation is that the most appropriate 
step to take at a given moment fails to occur to the formulator as a possibility. 
Congruously, the main advantage of a systematic procedure is reliability; 
if certain kinds of information are essential the system insures that these 
are obtained. 

We considered several ways of trying to combine system and intuition 
optimally. 

1. It is not uncommon that a system which has been designed 
intuitively is expected to be carried out rigidly and 
mechanically when put into operation. This seems to be the 
least promising combination. 

2. One can spell out the steps which might be expected to 
work best most of the time but permit the formulator to 
depart from this program on an intuitive basis whenever 
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FIGURE 1 



Cue 



Action by Formulator 



A symptom or need is identified 



Outcomes compete or are incon- 
sistent 

Outcome has hidden costs or 
consequences 

Outcome resembles a solution 







Outcome stated generally *■ 

Evidence of desired outcomes is *- 

agreed on 



Ask what outcomes are most desired 
from any change. 

Get priority ordering of outcomes „ 

Check consistency of outcomes with 
values. 

Ask why outcome is wanted. 

Ask for kinds of evidence of outcome. 
Assess present condition . 



Previous efforts have failed 



Disagreement an obstacle to 
action 



Ask why. Which factors are still 
present? 

Who or how many? Who controls 
decisions? 



Outcomes and present status clear — 

Solution ideas all conventional * 

Solution ideas exhausted — *• 

Questionable premise * 

Vague or rambling statement * 

Seemingly irrelevant comment * 

Perseveration on an idea * 



Get ideas for actions or solutions . 

Ask for wild ideas. Explore 
implications of extremes ("What would 
happen if...") 

Evaluate solutions, locate information 
gaps, pose research questions „ 

Challenge it » 

Restate simply and ask for verification. 

Ask for clarification of relation to 
the problem . 

Record idea visibly and ask for other 
ideas. Shift to a new element. 



Lack of rapport, noncooperation, 
conscious digression 



Re-examine purposes of inquiry. Find 
common goals. Point out useful 
functions of interaction. 



Information gap 



Problem definition looks 
complete 



Identify best information sources. 



Weigh cost of search against value 
of information. 



*■ Apply criteria to determine 

what is lacking. 



his judgment indicates some better course of action. For 
example, the flow diagram in Figure 2 shows a fairly 
simple set of steps to be followed to define the key 
elements of the problem, with each obtained element 
serving as a cue for the next step. 

3 0 Instead of being guided by the "best procedure" the 
formulator could have in hand a check list of cues and 
appropriate steps as illustrated in Figure 1. He could 
use his intuition freely but have the check list to refer 
to as a reminder of some important step that he might 
otherwise miss. However, in practice we found the 
check list too demanding and distracting from the task 
of interpreting information obtained from the educator 
in an interview situation. If the formulator could use 
his intuition freely during an interview with an educator, 
subsequent to each interview he could at his leisure 
examine the information obtained to date and compare it 
with a checklist of needed information. This comparison 
could then be used as a basis for structuring the next 
interview. If the formulator were sufficiently skilled 
or the check list sufficiently short, the time intervals 
might be condensed so that the whole process was 
accomplished in a single interview with an occasional 
break for the formulator to review his check list. 

One of the dangers of any systems such as those just described is that, 
through repeated use of a fairly prescribed set of procedures, the formulator's ideas 
and expectations about educational problems will be routinized into fairly 
narrow channels. A routine formulation process may thus miss some key leads 
to a new kind of problem -defining activity. This brings to light a paradox 
concerning who is the driving force behind the whole inquiry. If an outside 
formulator initiates the process, as we necessarily did in this project, the 
educator experiencing the need naturally expects him to structure and guide 
the process on the basis of his expertise . Hence the educator tends to play 
the role of passive provider of information rather than equal partner in 
formulating a problem. Yet the educator presumably should define the purpose 
of the inquiry, and the purpose should guide the process. The paradox is 
aggravated by the fact that educational problems are so diffuse and over- 
lapping. For example, in spelling out what outcomes are desired, the 
educator may name objectives which are quite important to the system but 
which are not closely related to the need initially identified. If the 
formulator notes this discrepancy he can ask the educator whether he is 
off course or really wants to shift to a different problem that is perhaps more 
important. It is especially easy for them to get off track if the outside 
formulator has initiated the whole inquiry since he is directing the process, 
yet does not know what the problem is. On the other hand if an educator 
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FIGURE 2 



FLOW CHART OF PROBLEM FORMULATION STEPS 
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initiated the contact because of a pressing need he wished to have formulated, 
it is much less likely that he would allow the inquiry to get off course, 
especially if he were paying for the service. Zeroing in efficiently on a 
problem may therefore be enhanced if the person directing the process 
establishes the objectives as quickly as possible. 

Systems and Social Patterns of Problem Formulation 

What is needed, it seemed from our exploratory interviews, is a system which 
will obtain the five components of information described above in any order 
or manner which permits the participants to make the maximum use of their 
own ingenuity to relate the problem components into an integral whole. 

It appears that the bulk of the formulator's work consists of gathering 
information. The ways in which he thinks about this information and helps 
to integrate it into a defined problem may be as critical but are not as 
time-consuming as collecting and verifying the information. If the 
information desired is recorded or already known by someone it is typically 
much more economical to obtain it by inspecting a document or asking 
someone than by trying to observe the educational environment directly and 
infer this information from observation. Even when personal bias is likely 
it may be more efficient to ask two or several individuals for the 
information independently rather than to obtain it by observation. The 
best strategy would seem to be to reserve for direct observation those 
information needs which the educators of the system cannot supply. 

In imagining optimal problem formulation systems we had envisioned the 
possibility of trying out a complex multi-stage attack on a problem in 
which individual and group interviews, direct observation, literature and 
document search, and other activities were used in series or combination 
with intermediate steps of summarizing and evaluating the data. Trying 
out a system of such a complexity turned out to be clearly beyond the 
scope of the present study. Our tryouts and evaluation focused on the 
initial intensive effort to formulate problems through verbal interaction. In 
retrospect this did not change the essential qualities of the problem 
formulation. Extended searches for information which might have been 
incorporated in a more complex system were rather considered as 
alternative actions to be weighed in a decision following the problem 
definition. 

With thought and verbal interaction as the main activities of problem 
formulation there still remains considerable freedom to choose among social 
patterns. Our early explorations had consisted of individual interviews. 

Small groups structured in various ways present feasible alternatives. 
Brainstorming groups, in which ideas are recorded as rapidly as they can be 
thought of by members of a group and no criticism or evaluation takes place 
during the idea generating session, has been widely promoted (Osburn, 1953; 
Clark, 1958). However, carefully controlled studies (Taylor, et al., 1958; 
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Dunnette, et al., 1963; Parnes, 1963) do not support the superiority of group 
brainstorming to the summed individual efforts. In all, the results from 
comparisons of small group and individual problem solving are not conclusive 
(Zagona, et al., 1966). Individual interviews may also permit greater 
confidence and security in revealing socially touchy but important factors in 
a problem. Maier (1967) suggests the following additional disadvantages of 

groups: 

1. Social pressure for conformity may stifle good ideas. 

2. One individual may dominate the group. 

3. Winning an argument may be valued more than the defining 
of the problem. 

What then might be the advantages of small group problem formulation over 
individual interviews? The main advantage suggested by our explorations is 
that in a group it is easier to focus interaction on goals and problems of 
the educational system rather than those of the particular individuals who 
are participating in the formulation. In individual interviews discussion 
frequently and naturally seemed to drift toward the concerns of the 
individual educator and his role in the system. Sometimes these personal 
concerns translated easily into system needs and sometimes they did not. 

Other advantages of groups are suggested by Maier as follows: 

1 . A group brings a greater sum total of knowledge to bear on 
the problem at one time than do the individuals working 
separately. 

2. Individuals tend to get into ruts in their thinking. Group 
members can knock each other out of these ruts. 

3. Participation in working on a problem increases acceptance 
of decisions about the problem. Thus a problem definition 
adopted by a group is likely to have better acceptance at 
least within the group than a definition which springs full- 
blown from a single individual. 

4. Members of a group are likely to comprehend a decision 
(or a problem definition) better as a result of group 
discussion of the problem. Individuals working separately 
would not get the benefit of this discussion. 

In many respects the success of group problem formulation probably depends on 
the skill of the group leader. If he can keep the group task-oriented and 
free of personal or interpersonal concerns the product is likely to be much 
better. Some of the personal or interpersonal concerns most likely to get 
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in the way are proving one's own competence, trying to please the management, 
trying to avoid responsibility, scapegoating and veiled aggression, and fear of 
failing or appearing foolish. A critical difference here is that conflict and 
dissent be treated as a source of useful ideas rather than as occasion for hard 
feelings. Hoffman, et al. (1962) found that in group problem solving the 
quality of solutions was greatly increased by encouraging subordinate members 
to openly express and discuss their disagreements with the suggestions made 
by an authority in the group. 

The leader can keep the group on track and minimize tangential discussion. 

On the other hand if he is too directive he may inhibit the contributions 
that individual group members could make. Maier (1967) concludes that 
the leader should serve mainly to integrate information and enhance 
communication, and that his direct contribution of content or evaluation 
should be minimized. 

The product of a group problem formulation depends ultimately on the quality 
of the individual member's contributions. Involving the educators in the 
task who are best able to define the particular problem is doubtless a 
critical factor. In order to bring the maximum amount of relevant information 
to a group effort, it seems wise to compose the group of persons who have 
specialized knowledge or familiarity with the problem. There is also 
evidence (Hoffman, 1959; Tuckman, 1967) that groups which are heterogeneous 
in other respects may perform better. A review of other variables relevant 
to group effectiveness in problem solving has been provided by Zagona, 

Willis and MacKinnon (1966). 

Experimental Comparison of Three Types of Group Problem Formulation 

The small group situation appeared to have enough advantages for problem 
formulation that we decided to compare three basic types of group problem 
formulation. These types differed on the extent to which the task was 
structured to obtain the rational elements of a problem described previously, 
and on whether the process was directed by an "outside expert" formulator 
or not. In one condition the groups had neither structure nor direction, in 
another the groups had structure but no direction, and in a third the groups 
were both structured and directed by an outside formulator. All problem 
formulations were evaluated later by other educators from the same school 
system . 

Method . In one elementary district, the group problem formulation procedures 
were first tried out and informally compared with a "detective” technique 
involving individual interviews and classroom observations by an outside 
formulator. Very similar kinds of problem data were obtained by the two 
approaches. We then made arrangements with three school districts (two 
unified, one secondary) to try out and compare the three group procedures. 

The districts were asked to participate only if they had actual educational needs 
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which they would like to have better defined for the benefit of the school 
district, rather than just as a matter of research interest. The administrative 
leaders of the district therefore chose problems to be formulated which were 
actual needs of the district. 

In each district the experiment was replicated twice so that there were six 
sets of three groups in all,, and three persons in each group. In two 
districts the three groups which were compbred analyzed the same problem, 
and each set of three groups took a different problem. In the other district, 
method differences were confounded with problem differences in that each 
of the three types of groups analyzed a different problem. In the latter 
district administrators and district staff made up one set of three groups 
while teachers made up the other set, whereas in the other districts each 
group within a set represented a heterogeneous combination of three 
members of the faculty or staff. The three districts, widely separated 
geographically, represented three types of population densities: suburban 
metropolitan, small city, and rural. 

All groups were given a brief general orientation which included the 
following: 

1. Verification that each member was interested in the problem. 

2. A brief description of the project from our point of view. 

3. Assurance of confidentiality where requested. 

The three types of groups met in separate rooms and were given two hours 
for the task of problem definition. Actual times varied from one and a 
half to two and a quarter hours. Each group was instructed as follows: 

Group A (unstructured). The group was told to organize and carry out the 
task in any manner they saw fit. A booklet of blank pages for one member 
to record the group product on indicated only that problem definition 
included consideration of possible solutions. Each member was handed a half- 
page of printed guidelines which is shown in Appendix C. They were told 
that we did not intend the recording secretary to be considered the leader 
in any other sense. The group was observed from time to time by an 
experimenter who was allowed to answer questions about method but not to 
make any other contributions. 

Group B (structured). These groups, like the A groups, were assigned a 
recording secretary (whoever volunteered) but not a leader. They were asked 
to focus their activity on providing the following basic elements of a problem 
definition: 
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1. The need as initially identified. 

2. Desired outcomes, and for each outcome the reasons it is desired 
and what evidence would be acceptable that it was achieved. 

3 . Previous actions . 

4. Limiting factors. 

5. Possible solutions with pros and cons. 

6. Possible immediate steps. 

The actual form on which this information was recorded and the guidelines 
handed to each member for using it are shown in Appendix C, The group 
was observed part of the time by a nonparticipating experimenter as 
described for Group A. 

Group C (structured -directed) . These groups sought to provide the same 

elements of a problem definition as the B groups. However in addition to 
the three regular group members one of the experimenters served as group 
leader and recording secretary. He concentrated mainly on eliciting comments 
from all group members, especially those in subordinate roles. Everyone's 
ideas were treated equally. He also summarized and recapitulated what was 
said by the group and contributed content himself. 

The hand-written notes taken at each group meeting were elaborated into more 
complete statements and typed. This editing process required one to three 
hours for each group protocol and generally resulted in longer problem 
definitions for the structured -directed groups (C) because the experimenter 
could recall more of what took place and of course could elaborate his own 
thoughts more easily than those of others. A sample set of three problem 
definitions, titled "Problem Analyses", one from each treatment group, is 
shown in Appendix D. The typed problem definitions were mailed to each 
of the participants, each person receiving the definition produced by his own 
group. In addition, two top-ranking members of the staff from each district 
who did not participate in problem formulation received the problem definitions 
from all three types of groups. The definitions were numbered and stacked in 
random order and the experimental treatments and names of participants were 
not identified, so that evaluations would be blind. 

Evaluation Results. Each participant and each independent evaluator was 
asked to study the problem definition^) received and then to. evaluate each 
one frankly by answering the questions listed in Table 2. Frequencies of 
each type of response from the twelve questionnaires completed by independent 
evaluators are shown for each question. The open-ended questions (*4 and $ 7 ) 
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TABLE 2 



Frequencies of Different Responses by Independent Evaluators 



Ideas About What Actions to Take Next 



1 . How many different actions were considered as possible solutions to 
this problem for the first time in this analysis, as far as you know? 



Treatment 

Group 



B 

C 



Number of Actions 

2 3 4-5 6 up 



1 



1 



1 



Pooled 

Sum 

11 

6 

30 



2. (If the answer to No. 1 is not zero) How many of the actions in 
No. 1 are at least as promising, in your opinion, as the best 
action suggested before this analysis began? (Consider both 
quality and cost in your answer.) 



Treatment 

Group 



B 

C 



B 



Number of Actions 

2 3 4-5 6 up 



1 



1 



8 



1 



Pooled 

Sum 

8 

5 

11 



3. Has the analysis led you to change your evaluation of the relative 
merit of different solution ideas? 



Yes 

1 



No 

11 

12 



4. Do you know of other promising actions not mentioned here? If so, 
please list them beiow: (Number who listed one or more actions for 

each treatment group: A -7, B-6, C-6). 
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TABLE 2 (Continued) 



Definition of the Need or Problem 



5. Disregarding ideas on "what actions to take”, which phrase below 
best describes the effect of this analysis on the definition of 
what the problem is? 

Treatment Group 

A B C 

2 11 much clearer now than before 



8 



a little clearer now than before 



about as clear now as it was before 



0 



less clear now than before 



6. Has the analysis yielded any new worthwhile ideas of the following 
kinds? 

Treatment Group 



A 

2 

2 

2 

0 

1 

1 

0 



B 

4 

1 

1 

1 

2 

0 

0 



c 

7 

3 

5 

3 

2 

3 

1 



What the objectives of any change should be 
Underlying reasons for wanting a change 
How this particular need relates to other needs 
Causes of the problem 

How to evaluate the success of any action taken 
Reasons why previous actions may have failed 
Other 



8 



9 



24 



TOTAL 



TABLE 2 (Continued) 



7. List all other things which need to be known or clarified before 
a sound decision could be made as to what step to take next. 



A 

B 

C 



Number of Things Listed Pooled 
0 1 2 3 Sum 

4 4 13 9 

5 2 14 15 

4 2 4 2 16 



did not differentiate the groups at all. Answers by the independent evaluators 
to the other five questions clearly favored the structured -directed (C) groups. 

On all but one (^2) of these five questions, differences among the three groups 
were statistically significant at P< .05 or beyond when a chi-square test was 
applied to the frequencies pooled across all independent evaluators. On 
Questions 1, 2, and 5 Group A was superior to Group B (no separate statistical 
test) while on Questions 3 and 6 Groups A and B were rated essentially the 
same • 

About two-thirds of the participants in the problem formulation groups returned 
completed evaluation forms. Although the participants rated their products 
somewhat higher across the board than the independent evaluators as might be 
expected, the relative standings of the three treatment groups were in 
substantially the same order for the participants as for the independent 
evaluators. 

The main result seemed to be that effectiveness of our structured approach to 
obtaining the basic elements of the defined problem is dependent primarily on 
the direction and contribution of an experienced problem formulator. Without 
this direction and with educators untrained in problem formulation our structured 
technique was probably actually slightly inferior to letting a leaderless group 
attack the problem in a completely freewheeling unstructured way. This 
experiment cannot tell us whether the directed groups' superiority was due 
mainly to the formulator's direction of the process or to the content he 
contributed, or both. Nor can we say how other trained problem formulators 
would fare in comparison. 

Looking again at the absolute response levels, especially for Questions 5 and 6, 
it is clear that none of the problem formulation methods worked miracles. On 
Question 5 most evaluators of Group C definitions indicated that the problem 
was a little clearer. With the Group B definitions, the problem was about as 
clear as before, and with Group A definitions, about half way in between. 

On Question 6, "What the Objectives of Any Change Should be" was the 
only category on which over half of the independent evaluators indicated that 
new worthwhile ideas were suggested. Perhaps this degree of accomplishment 
is about all one can expect from three people in two or three hours of initial 
effort on a problem, no matter what technique is used. In the absence of some 
dramatically better technique, how long would it take to add significantly to 
the completeness of the problem formulation we obtained? Quite a while, is 
our impression. In one of the three districts where the experimental data were 
collected, in addition to the time spent in regular data collection, approximately 
one day was spent interviewing others who were familiar with the two problems 
analyzed and observing classrooms. These experiences largely confirmed what 
was learned during the experiment itself but did not add important new dimensions 
to the problem definitions. It seems that what is already known or thought out 
by the educators in the system can be spread out on the table fairly quickly, 
but digging out new information and designing new approaches are slow laborious 
processes. 
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Implications for Training. The contrast in results between Groups B and C 
suggests that carefully designed training in problem formulation might substantially 
improve educators' problem defining skills . We examined the content of the 
Group B protocols to see whether omission or misinterpretation of one or two 
elements of a definition might account for their lower evaluations. No one 
element seemed to be grossly misunderstood, but many of the groups paced 
themselves badly and had too little time for the later elements. The 
participants did supply each of the kinds of information asked for in Group B, 
although it was sometimes stated too generally and occasionally was only 
marginally relevant to the outcomes desired. "Evidence" tended to be long term 
consequences of achieving an objective rather than immediately applicable 
measures of it. But this too is valuable information and this subtle difference 
was not stressed in orienting the groups. In the unstructured A groups, nearly 
all of the information provided could be categorized as either causes of the 
problem or possible solutions. The sample protocols in Appendix D illustrate 
some of the above differences. 

Training to use a structured problem formulation technique such as we employed 
should probably focus on at least two skills: reviewing specifics and formulating 

research questions. If specific outcomes sought were reviewed in detail as a 
stimulus to thinking of possible solutions, the tendency to address the problem in 
terms too general might be reduced. It was probably largely the fault of the 
way our form structured the procedures that the resulting formulation did not 
contain more research questions to be answered. Perhaps there should be an 
explicit step in the procedure which requires the participants to locate 
critical points of ignorance. Such points might indicate which outcomes are 
farthest from being achieved and what are the pivotal features of any proposed 
solution. More emphasis on posing such specific questions would probably 
lead the participants to propose immediate steps of search or research which 
would provide a better basis for later decisions between alternative solutions. 

Promising Directions 

Whatever approach is taken to problem formulation, two processes seem to be 
fundamental in it: generating ideas, and evaluating ideas. The conditions 

which facilitate these two activities appear to be fundamentally different. 

Idea generation is enhanced by a climate which accepts all contributions, 
encourages wild imagination, and takes any contribution as a stimulus for 
other constructive contributions rather than an object to be evaluated . 
Evaluation, on the other hand, seems to be helped by a realistic sense of 
responsibility for consequences, facing the facts in a hard-nosed way, and 
systematic procedures which insure that critical variables are accounted for. 

It would seem wise to separate these two processes temporally, but there 
is some question as to how much this can be done in problem definition 
without arbitrarily fragmenting the entire inquiry. As recommended in 
brainstorming and related group techniques (Maier, 1963), it may be wise to 
get all ideas for solutions expressed and recorded before any evaluation of 
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of solution ideas takes place. Would this same tactic work for the other 
elements of a problem definition such as outcomes, evidence, and present 
conditions? Or would the main result be to generate a lot more irrelevant 
content? 

If idea generation and evaluation can be separated effectively, perhaps the 
persons who serve these two functions should be chosen independently. We 
have no reason to believe that idea generation and idea evaluation are highly 
correlated skills. Idea generation and idea evaluation are somewhat analogous 
to divergent and convergent creative thinking respectively. Guilford, et al., 
(1961) found that interest in divergent thinking and interest in convergent 
thinking emerged as independent factors (slightly negatively related, in fact) 
from a factor analysis of 40 measures of temperament, interest and thinking. 

The distinction between idea generation and evaluation applies to need 
identification as well as problem formulation. That is, there are always 
various ways in which a school system can be improved. Generating ideas 
about such needs is one thing; deciding which have the highest priority and 
deserve to be formulated as problems is quite another. 

Although our studies were limited to data provided by educators, we highly 
recommend that any practical problem formulation system include students, 
parents, employers, scholars and other citizens. Lack of communication 
among these groups is the source of a good many educational problems in 
itself. As discussed earlier, there is no reason to suppose that younger 
persons or those having less status in the community will contribute any less 
to the task of identifying needs and defining problems. Amos and 
Washington (1960) found that students were better able to identify classroom 
behavior problems than were their teachers. For idea generation, the 
representation of more diverse viewpoints produces more cross fertilization of 
ideas as well as a greater number and variety. In evaluating ideas, all 
groups who are going to suffer the consequences of any action deserve to 
be in on the initial formulation, and this certainly includes all the groups 
mentioned above . 

How to implement a system which utilizes such diverse groups efficiently is 
certainly a challenge. Face-to-face meetings are especially hard to arrange, 
and though a certain number might be necessary, this number could be 
minimized by relying on other communication media such as mail and telephone. 
Systematic sampling of each population group seems preferable for evaluating 
ideas, but for idea generation public appeals for voluntary contributions should 
serve just as well. 

Our results suggest that the process should be organized and directed by 
someone who has ticquired special skill in formulating educational problems. 

The future availability in sufficient number of outside experts for this 
purpose is uncertain, so it is practical to consider who might perform this 
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function within the school district or county. The skills required would 
logically seem to fit the position of director of research and planning. An 
increasing number of school districts in the nation are including positions 
similar to this on their staffs. Often a great deal of time and effort in 
problem formulation could be saved by including as participants outside 
specialists and educators from other school districts who may already have 
struggled through similar problems. 

Having the problem identification and formulation process directed and 
organized by a specialized central facility which could serve a wide region 
has some advantages. Specialized expertise and knowledge could be called 
upon more readily and coordination among districts, counties and states might 
be facilitated. The competence and efficiency of the central facility's staff 
would probably be enhanced as their experience with a large number of 
problems in the region grew. There is considerable overlap among various 
educational problems in both content and structure. Sarason (1961) has 
discussed how cumulative experience with human problems leads to growth 
in skill in dealing with subsequent problems. 

On the basis of its experience and regular communication with other informa- 
tion storage centers, a central facility could compile a useful file of 
problems. The critical incidents which we have collected and classified 
could be part of such a file. Educational problem files could be used in 
several ways: 

1 . A consulting problem formulator could use them as a job 
aid, referring to them for ideas which might be applicable 
to formulating a problem in a particular district. 

2. If problems, solutions and other material pertaining to 
each problem were cross indexed, an educator wishing 
to find out more about a particular problem could 
retrieve useful information from the central facility's 
file 0 

3. Periodically a list of problems encountered could be 
reproduced and mailed to all school districts in the 
region. Educators could then scan this list for new 
ideas as to what should be improved in their school. 

They could then request more information as described 
in No. 2 and/or proceed to get the problem formulated 
more clearly in their own district. 

We have dealt mainly with a situation where a need is identified and the 
problem formulation effort concentrates on that single need. But there is 
much to recommend in a multi-problem approach. Proposed actions typically 
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have a bearing on several problems, even though they may have been inspired 
by a single need® Treating the problems in conjunction provides a more 
rational basis for decision. If the total set of objectives within the educational 
system is spelled out and examined, priorities among problems may be established 
so that small problems are not beaten to death while larger ones go unattended. 
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APPENDIX A 

Form used to collect critical incident data. Only the 
first two pages of the form are shown. The third, 
fourth and fifth pages were essentially identical to the 
first page. On the final page general comments on 
their educational system were invited. 



Position 
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INCIDENT FORM 

First, would you try to remember a specific thing happening recently which made you 
feel that something about education here needed improving. Please recall one single 
event or one observation of some condition. 

When did this happen or when did you observe it? 

Where or under what conditions did it happen? 



Exactly what happened? 



(for the 3 questions below, write only information not already given above.) 
What led up to this event or condition? 



What, if anything, could have been done to prevent it? 



What other undesirable effects, if any, would you expect to result from this event 
or condition, or from its causes? 



A-2 



Now please recall and describe a recent event which made you feel that education 
here was working especially well in some way. 

When did this event or observation occur? 

Where or under what conditions did it happen? — 



Exactly what happened? 



What led up to this event or condition? 



What is the main good effect you would expect in the long run from events of 
this kind? — 
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2nd grade teacher A 5th grade student who is very intelligent finished his classwork 

quickly, becomes bored for lack of anything to do and is becoming 
a discipline problem. (There is a) lack of anything for the gifted 
child in our district. Offer some type of program for the gifted 
as well as the disturbed. 
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APPENDIX C 

Problem definition guidelines for Group A 
and guidelines and data form for Group B 
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GUIDELINES FOR PROBLEM ANALYSIS GROUP A 



General 

1 . The worth of any problem analysis depends mostly on your own effort and 
ingenuity . 'The Problem Analysis Form (A) which you have has only two 
general headings because we want to know what points you consider most 
relevant without our suggesting a definite structure. Feel free to organize 
your ideas and information in any way that seems to fit the problem. 

2. Please don't be too concerned about grammar or choice of words. What you 
write will be edited later. It is more important that you jot down all relevant 
information and ideas, however briefly. Be as specific as you can. 

3. We have provided more writing space than you will usually need. But use 
the back if you need more space . 

4. Since the problem at hand represents a real felt need for change in this school 
district, the results of your effort today will probably be used in planning 
actual policies or procedures. Therefore, anything you think is relevant should 
be included. 

5. You should allow about half the available time for considering possible 
solutions (the last two pages). 
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GUIDELINES FOR PROBLEM ANALYSIS GROUP B 



General 

1 . The worth of any problem analysis depends mostly on your own effort and ingenuity . 
The problem analysis form is intended to get your ideas explicitly spelled out and 
organized. It may or may not stimulate you to examine new aspects of the problem . 

We hope the form will supplement, but not interfere with, whatever problem skills 
you normally use. 

2. Please don't be too concerned about grammar or choice of words. What you write 
will be edited later. It is more important that you jot down all relevant information 
and ideas, however briefly. Be as specific as you can. 

3. We have provided more writing space than you will usually need. But use the 
back or margins if you need more space. 

4. Since the problem at hand represents a real felt need for change in this school district, 
the results of your effort today will probably be used in planning actual policies or 
procedures. Therefore, anything you think is relevant should be included, even if 

it does not fit any of our questions . 

5. You should allow about half the available time for considering possible solutions 
and next steps (pages 5 and 6). 

(The following headings correspond to headings of the problem analysis form .) 

Page 1 

The Need The purpose of this initial statement is to get everyone started on the same 
problem^ It is not meant to be a final or complete definition . Two or three minutes is 
all the time it deserves. 

Desired Outcomes Specifically what effect are you trying to achieve? Please reserve 
solution ideas and programs for Page 6. List here only the important results that any such 
action would hope to produce. If there is just one main outcome desired, leave the 
remaining lines blank. 

Page 2 

Outcome 1 On the top line abbreviate to a few words the outcome which is most important 
in meeting this need. (It should be numbered "1" on the previous page.) 

Please list all important reasons and values for wanting the outcome, however obvious they 
may seem . 

Answer the questions about evidence as specifically as ybu can . How could a measurement 
or judgment be made? By whom? 
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Page 3 

If on Page 1 you have listed only one main outcome desired, skip this page. Otherwise 
follow the saline guidelines as for Page 2/ 

Page 4 

Previous Actions Briefly describe each action taken in the past which was intended to 
improve the status of this problem . If the action was not successful , a few words about 
why it failed might be helpful . Include previous efforts to define the problem (e.g., 
group meetings, research, searches for information, etc.). 

Limiting Factors Note here anything that might hinder a solution, but also please note 
(either here or under Solutions) ways in which obstacles might be overcome or avoided. 
Some limitations which are taken for granted may lose their force when assumptions are 
questioned or new approaches are considered . 

^ge 5 

Possible Solutions It may help to review the information on previous pages both in 
getting new ideas about solutions and in evaluating them. Especially, try starting with 
one desired outcome and thinking of all the ways that this could happen, however absurd 
or impractical . Then see if these ideas can be made practical in any way . Please note 
in the left column as many different solution ideas as you can, however wild they seem, 
before you try to evaluate each one . 

If a solution is likely to achieve one desired outcome but not others, note this under 
’’Pros and Cons" . Also note there any good or bad side effects, hidden costs, and so on. 

In evaluating "Expected Effectiveness" you may group solution ideas together into 
"program” and rate the whole package. To make the rating scale finer you may combine 
two adjectives, such as "poor-fair" (P— F) or "excel lent —good (E-G). 

Immediate Steps This part should be based on a careful review of the outcomes, values, 
limiting factors, and solution evaluations. 
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PROBLEM ANALYSIS FORM B 



Name Position 

Schoo I 

Date 



The Need : What is the situation or condition in need of change, and what mainly causes it? 



Desired Outcomes: What important effect do you hope to result from any change? Forget how 
the change might be accomplished for now. Just write down what outcome or result would satisfy 
you . (If you name more than one outcome, number them in order of importance.) 

( ) 



Outcome 1: 



Why do you want this outcome? List the reasons and values. 



What would you accept as evidence that this outcome was achieved? 



Would the evidence above convince others concerned that the outcome was achieved? If not, 
what additional evidence might convince them? 



(Skip this page if there is substantially only one important outcome sought .) 






Outcome 2: 



Why do you want this outcome? List the reasons and values. 

r 



What would you accept, as evidence that this outcome was achieved? 



Additional evidence needed to convince others? 



Outcome 3: 



Why do you want this outcome? List the reasons and values . 



What would you accept as evidence that this outcome was achieved? 



Additional evidence needed to convince others? 
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- Previous Actions: What actions have already been taken to meet the need, or to provide 

background information? With what results? 



Limiting Factors: What will any proposed solution have to take into account? (circumstances, 
people, restrictions, etc .) 



Possible Solutions; On the left below, list alternative ways that the desired results 
might be achieved. In the center column, rate the expected effectiveness of each 
solution? "excellent" (E), "good" (G), "fair" (F), "poor" (P). On the right, 
note briefly the main "pros" and/or "cons" of each solution. 

Possible Solution Expected Pros and Cons 

Effectiveness 



Immediate Steps ; What are the two or three most reasonable alternative steps which might be 
taken next? Time demands, costs, information needs and the expected effectiveness of 
solutions are some of the things that need to be considered. 




APPENDIX D 

Examples of completed problem 
definitions for treatment groups 
A, B and C. The examples 
shown are all definitions of the 
same problem in a given school 
district. 
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PROBLEM ANALYSIS — Group A 



Brif f Initial Statement of Need: A number of teachers, many of whom are tenured, have reached 
a plateau in their professional growth and development. Ways need to be found to upgrade the 
competency and motivation for growth in these teachers. 

Background and Causes 

1 . Presently the system rewards are distributed on the basis of seniority, with the 
older teachers receiving most of the benefits . Many new teachers are asked 
to teach courses out of the area of their general competence, such as driver 
education and safety . This is thought to have particularly poor effects in the 
early stages of a teacher *s career when his enthusiasm is at a high level . 

2. Many teachers do not maintain and increase their knowledge in their subject 
area . Some of the causes for this might be: 

a. Lack of administrative machinery, such as salary incentives, 
supporting philosophy and policies, to aid the teachers in keeping 
up on recent developments in their field . Lack of communication 
regarding the availability of this machinery could also be a factor. 

b. Job descriptions are not sufficiently clear. 

c . Insufficient feedback as *o success of administrative programs and 
policies. 

3. Lack of ability on the part of the teacher to understand individual differences 
of students. Some of the factors contributing to this are: 

a. The generation gap. What aid could be made available to teachers 
to better understand students? The recent program on narcotics was 
quite helpful, but more is needed . 

b. Sheer numbers of students. It is difficult to treat 150 people per day 
as individuals. 

4. Inconsistency between overall school and classroom philosophies. These two 
should clearly coincide. 

5 . Lack of administrative consistency and firmness toward faculty. The 
administration does not always speak with one voice; sometimes it does not 
speak at all . 

6. Low morale among the faculty. 

7. The inherent characteristics of the teaching profession sometimes serve to 
attract those people who want to avoid a competitive atmosphere and thus 
are more likely to reach a plateau in their professional growth. 



8. The overwhelming and confusing nature of the bureaucracy. 

9. The mental "instability" of teachers as a professional class. 

Possible Solutions 

1 . The advantages in the system should be more equitably distributed between new 
and old teachers, particularly with respect to placement and teaching load . 

2. Upward communication and participation structures could be improved. There 
are presently three formal structures through which teachers can communicate 

to the administration and take part in decision making. These are: the Principal's 
Advisory Council, the Ad Hoc Committee and through Department Chairmen. 

These have improved the situation considerably . However, the availability of 
these channels should be better advertised and, more importantly, there should 
be feedback as to the use made of them, the outcomes, etc. The administration 
should create the awareness that communication lines are open and messages 
are welcome. There should be faculty participation in Board meetings. 

3. There is presently too much noise in the downward communication channels. 
Although information concerning policies and top-level bureaucratic functions 
is passed down, it is embedded in non-essentials and rumor. The result is that 
it requires too much effort on the part of the faculty to really know what's 
going on . Instruments of communication should be modified so that only 
essential information is communicated . For example, the Faculty Handbook 
could be considerably shortened and made an effective instrument for 
communication of policy. 

4. The administration should offer more support and firm guidance to the facfd'y. 

Often the feeling on the part of teachers is that the administration is 
indifferent. This, in addition to the lack of knowledge about the functions of 
the top-level bureaucracy, creates a dysfunctional we/they atmosphere. 

Increased support could take the form of classroom visitation on the part of 
administration; frequent, perhaps scheduled, one-to-one meetings between 
administrators and teachers; increased formal social interaction both in and 
out of school . 

5. Bureaucratic red tape should be trimmed to the essentials. 

6. The teacher/ student ratio could be lowered . 

7 . Various techniques could be used to help teach ere better understand the 
students both as a group and individually. Inservice programs are one device 
intended to bridge the generation gap which has been used with some success-. 

More programs of this kind are needed . Role playing techniques could be used 
to develop the teachers' perception of students and themselves. Also, more 
information could be made available to teachers on individual student backgrounds. 

8. Various steps could be taken to both prevent and effectively deal with teacher 
stress. Flexibility of teaching schedule, such as is found in the quarter system, 
might be very helpful in this regard. Also, there should be an accepting and 
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supportive attitude toward teacher stress on the part of peers and particularly 
administration* This would ideally include an attempt on the part of the 
administration to relieve some of the pressures on teachers by manipulating 
the situation in some way - such as removing problem students anc|/or extra 
responsibilities, etc. 

9. Selection and retention procedures also need to be re-evaluated. Some of 
the training techniques mentioned above could be instituted early in the 
teacher's career. If these proved to be unsuccessful in improving the teacher's 
performance, he could then be removed before tenure takes place. 
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PROBLEM ANALYSIS— Group B 



The Need: A number of teachers, many of whom are tenured, have reached a plateau in their 
professional growth and development. Ways need to be found to upgrade the competency and 
motivation for growth in these teachers. By competence is meant not only maintenance of 
knowledge in subject field, but also a teaching emphasis on thinking rather than memorizing, 
citizenship rather than grades. 

The causes of teacher incompetence or lack of motivation to grow are: 

1 . Boredom . A number of teachers, particularly those who have been teaching 
for a long period of time, have lost sight of their original goals. They tend 
to regard teaching as simply a job to be done . They are uninterested in help 
or somewhat reluctant to ask for it. 

2. Lack of success experiences with classes over a period of time . Teachers 
are frustrated with low achievers, discipline problems, and shortcomings 
in the curriculum. 

3. Group dissension among the faculty or between faculty and administration. 

The process of upgrading is difficult because: 

1 . Introduction of change in teaching methods is seen by the teacher as a 
personal threat. 

2. Scapegoating on the part of the teacher - “they won’t let me do such and such" . 



Desired Outcomes: What important effects do we hope to result from any effort to upgrade 

the effectiveness of tenured teachers. 

Outcome 1: Students would be capable of critical thinking. This outcome is a 
desirable end in itself. Also, graduates would be able to evaluate issues rather 
than rely on whatever store of "facts" they had acquired in school and would 
have the courage to take the consequent stand on issues . The society would 
thereby benefit by being more capable of dealing with its problems, such as 
poverty, civil rights, etc. 

Evidence of this outcome would be more sophisticated participation in civic affairs 
by graduates. Further, this participation would be dictated more by rational awareness 
than by self interest . 

Outcome 2: All students would be prepar ed to be productive and happy citizens. 

This outcome is a desirable end in itself. Students (citizens) would have a more 
positive self image, and the total society would consequently benefit. 

Evidence of this outcome (and of Outcome 4 below) would be as follows: 
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1 . Improvement in citizenship as well as academic achievement. 

2. Increased extra-curricular participation. 

3. Percentage of dropouts would decline. 

4. Attendance would improve. 

5. Number of graduates who return to visit the school would increase. 

6. Percentage of graduates that go on to hold positions of influence, such 
as public office, would increase. 

Outcome 3: There would be more rewarding experiences for teachers. This outcome 
is desirable because the feeling of success on the part of the teacher would lead to 
more success. It would also be conducive to the teacher's mental health. 

Evidence that this outcome was achieved would be: 

1 . Increased participation in voluntary assignments by teachers. 

2. On referrals to the Dean, teachers would not tell Dean what form 
of punishment should be meted out . 

3. Fewer complaints from teachers. 

Outcome 4: The administration could concentrate more on innovation and long term 
planning rather than solving short term teacher problems. This would lead to the overall 
impiovement of the entire educational system. 

One indication that this outcome was achieved would be that parents would be more 
involved in the educational process and would be more willing to support the educational 
system through bond issues. 



Previous Actions 



There have been programs of orientation and inservice training as well as the current curriculum 
development project directed toward this problem. These have had some beneficial effects 
but do not really reach those that most need improvement. The Principal regularly evaluates 
teacher growth, but up until now this evaluation procedure did not include tenured teachers. 
The main limitation on the effectiveness of previous programs has been their voluntary nature. 
Those that want to improve, or at least maintain, their competency have taken advantage of 
them; those that have needed them the most have not . 

Limiting Factors 

I . Teacher unwillingness to accept direction - especially from the administration, 
toward whom there is a strong negative attitude . 



2. Lack of desirability of meetings as a way of dealing with the problem . Teachers 
are usually too tired at the end of the day to attend meetings. 

3. Curriculum development effort cannot be counted on for necessary leadership 
in dealing with this problem. 

4. The dilemma created by teachers wanting more firm guidance, and at the same 
time resenting autocratic decisions. 

5. System red tape and legal restrictions. 

6. Small budget. 

7. Lack of understanding of the real issues on the part of those in power. 



Possible Solutions . 

1 . Attempt to individualize the supportive atmosphere for the teacher and then 
follow through with concrete support for mutually agreed upon action. This 
would optimally be done on a year-to-year basis. That is, have an individual 
conference with each teacher, talking over and deciding what particular 
kind of inservice training would be desirable for this teacher. If the machinery 
for this kind of training doesn^ exist, do what can be done, within reason, to 
implement it. 

Pro: This would give teachers a chance to communicate and give them 
the feeling that “someone cared” . 

Con: The ineffectiveness of college courses (if this alternative is chosen) 
in upgrading teacher competence . This is largely due to the fact that 
college courses are not sufficiently suited to meet the needs of our teachers. 
Possibly our own inservice program could incorporate more appropriate 
courses that were specifically designed to meet the needs of our teachers. 

2 . Institute course and other requirements that would apply to tenured as well as newer 
teachers . 

Pro: It would shake the tenured teacher out of his complacent attitude. 

It would make him appreciate what it is to be a student again. 

Con: Teachers would be upset over having to do it, and this will perhaps 
cancel out any positive effects. 

3. Encourage teacher involvement in industry so that he maintains knowledge of the 
state of the art, what is being required of employees, etc. 

Pro: Teachers will be better acquainted with what skills are currently in 
demand. 



o 
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4. Some form of student evaluation of teachers, though preferably not through 
student-designed evaluations. 

5. Case study of students to determine why one teacher has trouble with a particular 
student and another has no difficulty with him. 

Pro: Teachers often find things that they are doing ’’wrong" . 

Con: This approach has been somewhat overworked recently . 

6. More teacher/ community interaction. 

Con: The courteous atmosphere which would prevail would preclude 
any real gain from such involvement . 

7. Intradepartmental evaluations of teachers. 

8. Better methods of teacher evaluation. 

Pro: Would enable the administration to discover some of the causes 
contributing to a teacher's general incompetence or complacency. 

9. Passage of the bond issue . 

10. More dynamic leadership from top-level administration. 

11. Teacher transfers . 

12. Create a position for someone to exclusively attend to teacher development. 
Immediate Steps 

1 . Identification of those teachers who are in need of improvement - by small group 
discussion and classroom observation, including those of tenured teachers. 

2. Determine what steps can be taken to improve those teachers. 

3. Support the teacher in these corrective measures if such exist. If the teacher still 
does not improve, counsel him out of teaching or transfer them, either to another 
school or to a less critical spot. 



PROBLEM ANALYSIS - Group C 



The Problem : How to upgrade the effectiveness of teachers, especially teachers who have 
tenure . 

Desired Outcomes : What important effects do we hope to result from any effort to upgrade the 
effectiveness of tenured teachers? 

Outcome h More students can perform the skills and meet those objectives w hich 
each course is designed to achieve. This outcome shou Id lead to more successful 
entrance by graduates into careers or colleges of their choice. Thus, evidence 
of achievement of this outcome could be higher grades by graduates of high school 
while they were in college, more reported satisfaction with their careers by 
graduates and perhaps even better performance ratings of graduates by their work 
supervisors. 

To the extent that grades in high school are determined by achievement of 
specific objectives, such grades would provide more immediate evidence of 
this outcome . Outcomes 3 and 4 in themselves provide indirect evidence of 
achievement of this outcome. 

Outcome 2; For every course of instruction, clear objectives stating what 
students should be able to do as a result of instruction are spelled out in detail , 

Measures of achievement of these objectives are the basis for grading students and, 
more importantly, provide the standard against which teachers' effectiveness can 
be judged. It is important to note that this does not imply that students are 
expected to reach the same final level of achievement in a course no matter what 
their initial state of knowledge is on entering the course. Effectiveness of teachers 
would be judged in terms of progress made by students from their initial state of 
skill on entering the course, however low or high this level of achievement may be. 

This would require some form of pretesting as a regular procedure for every course 
or unit. 

This outcome would enable all concerned to understand clearly the criteria against 
which teaching is evaluated so that evaluation and promotion might be accomplished 
fairly. It would also provide a better basis for revising the curriculum. 

As evidence that this outcome was achieved, one could look at final course exams 
and see whether the questions are geared specifically to course or unit objectives. 

Outcome 3: Students are more enthusiastic about school. This outcome is a 
desirable end in itself and also leads to more learning and greater interest in 
issues of social importance on the part of students. 

Objective evidence of this outcome might be the number of voluntary questions 
and comments made by students. Also when asked what he was doing, a student 
might give a more intense, involved and elaborate response. Most teachers feel 
that enthusiasm can be judged fairly reliably simply by observation of students in 
clrss. 
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Outcome 4: Parents are more satisfied with the school system. This is evidenced 
by more positive and less negative feedback from parents to the school . 

Outcome 5: Teachers are more involved/ aroused and enthused themselves in 

the educational process and in their own subject area . 

Previous Actions 

[ l • Inservice workshops have been conducted to upgrade teacher effectiveness. 

Prevailing opinion seems to be that they were not too effective. Curriculum 
[. development workshops in the summer have apparently been good for the 

| teachers doing the writing, but other teachers frequently ignore the results 

[ of these efforts . 

I 

jf 

l 2. There are salary scale incentives for advanced education by the members 

l of the faculty. However, college credits do not seem to correlate highly 

with real progress in improving teaching effectiveness, so that this solution 
f is of I imited success . 

Limiting Factors: What will any proposed solution have to take into account? 

1 • Many of the teachers who have been in the system for a long time may not be 
receptive to the "prior objectives". approach to planning instruction, as 
l spelled out under Outcome 2. 

f 

l 2. Teachers who seek tenure itself as a goal, perhaps for the sake of security, 

i often are less concerned about the effectiveness of their teaching and less 

involved in their subject area. 

f 

l 3, Asa result of the previous history of conflict between teachers and administrators 

i over the issue of merit pay, any plan of action would have to include a special 

| effort to overcome strong opinions associated with the idea of merit pay. For 

example, teachers may fear that administrators will not give teachers the power 
to change the very things which are used as criteria in judging the teachers' 
l effectiveness . 

4. There may be discrepancies between teachers and administrators as to what are 
the main objectives. 

I 5. There is some resentment over the gap in pay between teachers and administrators. 

■v 

I 

I Possible Solutions 

| — 

$■ 

I 1 • Base the promotion of a teacher and other professional rewards on the achievement 

j of his students and the teacher's competence in his own subject area. (Solution 

| ideas 2 through 5 below elaborate possible means for implementing this idea.) 

i 

\ 
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2. If teachers are to be judged and promoted on the basis of what their students 
achieve, there must first be a clear agreement as to what the particular 
students in a teacher's charge should be expected to achieve. This means 
that objectives for the class and for a particular student in the class must 
have been spelled out clearly ahead of time. Further, the degree of attainment 
of these objectives, or percent of students expected to achieve at a given level, 
should be established as a basis for rating the success or effectiveness of the 
teacher. If objectives are not spelled out and agreed upon, the evaluator 
may have different standards of effectiveness from the teacher and the resulting 
evaluation will not be fair. Even if the teacher and supervisor bcdieved in 
the same general objectives, failure to spell out these objectives in detail 
might result in evaluation based upon different judgments as to specifically 
what achievement of these general objectives means. Therefore, a first 
requirement for promotion on the basis of effectiveness is for objectives of 
instruction to be spelled out in detail . This could be accomplished through the 
faculty of the district itself being given time to spell out these objectives 
and/or through borrowing, editing and selecting objectives developed by other 
projects, school districts or states. This approach does not require that objectives 
for a given course and age level be uniform throughout the district or even 
within a school . If a particular teacher has certain unique objectives that he 
and no other teacher, wishes to accomplish, then his effectiveness should be 
judged on the basis of those objectives to the extent that they are represented 
in his course. If on rare occasions the teacher and the supervisor cannot agree 
as to the appropriateness of objectives for a course, some sort of consensus or 
compromise should be worked out in advance so that evaluation may be made on 
a fair basis. Since unique objectives of a particular teacher fend to be rare, 
a practical procedure would be to spell out a specific set of objectives which 
suit most of the faculty members. Any given teacher should then agree that 
those are appropriate objectives or pick out exceptions and in place of those 
exceptions spell out the unique objectives that he would like to achieve instead. 

Once the objectives are spelled out, the next major prerequisite to fair 
evaluation is that adequate measures of achievement of these objectives be 
agreed upon. Again, the teacher's own efforts to create measures could be 
combined with measures developed by other districts or projects. Once the 
measures are agreed upon there remains the problem of norms as to what degree 
of attainment or progress in a year's time should be expected of students at 
different levels in trying to reach these objectives. If normative data of this 
sort are not available, it may be best to delay merit ratings based on such measures 
for two or three years until enough data are collected to provide a baseline 
expectation for what students with different entering proficiencies should be able 
to achieve . 

Once promotion is clearly based on measured achievement of specified 
objectives, teachers will naturally have the incentive to design their instruction 
to achieve these objectives in the best way possible. Some evidence that this 
does take place may be seen in the experience with use of the advanced placement 
test in the district. Apparently the orientation toward accomplishing specific 
skills measured by these tests helped focus the analysis and revision of high school 
English courses in the district. 
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If progress toward objectives is to be used as a basis for rating teacher effectiveness, 
there must be measures of the initial state of knowledge of students upon entering 
the course or upon entering a given unit of instruction . This requires the use of 
some form of pretesting before a unit or a course in order to establish the level of 
each student. 

3. In order to increase the effectiveness of all teachers and to reduce the variation in 
level of accomplishment of students entering the course, coordination on specific 
course objectives should take place between the high school and the elementary 
districts feeding it. If this does not take place through unification, then small 
standing committees in each subject matter might be given released time to insure 
this coordination of specific objectives. This would also give each student a 
feeling of greater continuity . In order to give this task of coordination of 
objectives some overall comprehensiven *ss and order, it would be necessary to 
create or adopt a general structure within which all subject matters could be 
related across all age levels. 

4. Administrators, especially principals, might be rewarded and promoted on the 
extent to which they create a climate conducive to learning. Such an 
objective might be judged on the basis of actual progress of students toward 
specific objectives and also on the satisfaction of teachers, students and parents 
that sue 1 * a climate is present. The main avenue through which administrators 
may accomplish this is through district and school policy which sets higher 
priorities on effective teaching than on some of the traditional concerns such 

as attendance, tardiness and other logistical matters. Basing promotions on 
accomplishments of specific objectives by students rather than upon these 
secondary criteria would naturally tend to promote a climate for learning. 

5 . Regarding the outcome of getting more teachers more involved in their own 
subject fields, to some extent this may result from rewarding effective teaching, 
in that how much students learn depends partly on the teacher's own competence 
in the subject area. However, additional rewards for advancement in their 
subject fields could be provided. The present policy of pay increments for 
college credits is only limited in effectiveness in that credit in college courses 
is widely perceived as not being correlated very highly with advancement of 
competence. Therefore, perhaps the rewards should be for the intended effects 
of these courses rather than for the mere taking of a course or workshop in itself. 

For example, teachers might be given tangible rewards for suggesting new 
objectives which are agreed to be worthwhile by most other members of the 
faculty in that subject area, or for new teaching methods which when tried are 
thought worthy, or for having convinced other facu'ty members of their competence 
in special subfields to the extent that they are recognized as a good potential 
resource person for particular subject fields. 

Perhaps the safest basis for judging advancement of a teacher's competence in 
a subject area is the performance of his students. If in addition one wishes to 
try to measure more directly this type of competence, it may be wise to bring in 
outside experts or objective measures of some kind . For example, master teachers 
from other districts might judge the promise or effectiveness of innovative methods, 
materials or objectives, although difficulties may arise in negotiating contractual 
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relations with teachers outside the district. Another possible approach is to 
ask nationally recognized experts what are some of the recent developments and 
ideas in that subject area which a good teacher should be aware of. This could 
then provide the basis for a "current events" test of teachers* awareness of 
current developments. 

6. There might be more flexibility in the progression hierarchies and rules for 
promotion set up by the district. The time limits on steps through which a 
teacher progresses may put too low a ceiling on rate of advancement for 
teachers who are especially competent. The provision for especially competent 
teachers to skip steps might provide additional incentive, as might the provision 
for transferring more than seven steps to a new district. There might also be 
separate progression hierarchies for teachers, administrators and perhaps even 
counselors. The fact that the highest paid teacher is paid less than the lowest 
paid administrator seems to put an arbitrary ceiling on the incentive for 
especially competent teachers. This policy may also force the use of highly 
paid administrative personnel for functions which lower paid employees could 
well perform . Separate progression hierarchies would have the added advantage 
that teachers who are especially competent would not have to leave their 
occupation in order to progress in salary. Rather our best teachers would be 
kept teaching and rewarded for improving their skills rather than changing to 
a different occupation for which they may have little talent, i .e., administration. 



Possible Immediate Steps 

1 . The district staff might meet to jointly consider this problem and map out 
some of the alternatives. 

2. The Board limitation on transferring steps, that is, the limitation to seven, 
could be reconsidered. 

3. A new plan for salary incentives based upon demonstrated competence as 

well as formal degrees, including a master's degree perhaps, could be outlined. 

4. Administrative changes which would permit teachers to have a greater voice 
in setting objectives and school conditions necessary to meet these might be 
explored and discussed . 



